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Abstract 

Background: The patient assessment of chronic illness care (PACIC) is a promising instrument to evaluate the chronic care experiences 
of patients, yet additional validation is needed to improve its usefulness. 

Methods: A total of 1941 patients with diabetes completed the questionnaire. Reliability coefficients and factor analyses were used to psy- 
chometrically test the PACIC and PACIC+ (i.e. PACIC extended with six additional multidisciplinary team functioning items to improve 
content validity). Intra-class correlations were computed to identify the extent to which variation in scores can be attributed to GP practices. 

Results: The PACIC and PACIC+ showed a good psychometric quality (Cronbach's alpha's >0.9). Explorative factor analyses showed 
inconclusive results. Confirmative factor analysis showed that none of the factor structures had an acceptable fit (RMSEA>0. 10). In addi- 
tion, 5.1 to 5.4% of the total variation was identified at the GP practice level. 

Conclusion: The PACIC and PACIC+ are reliable instruments to measure the chronic care management experiences of patients. The 
PACIC+ is preferred because it also includes multidisciplinary coordination and cooperation — one of the central pillars of chronic care 
management — with good psychometric quality. Previously identified subscales should be used with caution. Both PACIC instruments are 
useful in identifying GP practice variation. 
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Background 

Chronic care management for patients with diabetes 
has changed in recent decades. Initiatives such as 
multidisciplinary protocols, pro-active care plans, and 
additional education have been introduced [1, 2]. Most 
of these initiatives are in line with the widely adopted 
chronic care model (CCM) [2]. The CCM is promoted 
as a guide to improve chronic care to realise patient- 
centred care in which problems such as fragmentation, 
guideline non-adherence, and restricted self-manage- 
ment are limited [3, 4]. Because patient-centredness 
is becoming more and more important in chronic care, 
it is of importance to measure the chronic care experi- 
ences of patients [5]. Patients' chronic care experience 
is positively related to other aspects of health care 
quality, including their engagement with and adherence 
to provider's instructions as well as clinical processes 
and outcomes [6]. Moreover, patients' experiences can 
be used for quality improvement or even as a bench- 
mark tool [5-9]. 

Several instruments exist to measure patient's chronic 
care experiences [9]. The Patient Assessment of 
Chronic Illness Care (PACIC), which measures the 
extent of alignment of chronic care with the CCM, is one 
of the most promising instruments to measure patients' 
chronic care experience [8-11]. Previous studies sug- 
gest that PACIC scores can be used to direct quality 
improvement programs [8-11]. 

Notwithstanding the promising results from previous 
studies, additional validation of the PACIC is needed 
to improve this instrument [9, 11-13]. First, it is still 
unknown which PACIC subscales are appropriate to 
use. Previous validation studies, with the exception of 
the recently performed validation by Gugiu, used inap- 
propriate methodological tests for PACIC's ordinal data 
structure [12, 13]. Gugiu validated the PACIC with a 
modified response scale to avoid an ordinal structure; 
however, this modified response scale was unsuccess- 
ful [12, 13]. 

Second, the PACIC is assumed to measure the extent 
to which chronic care, for instance diabetes care, is 
congruent with the CCM. However, not all components 
of the CCM are fully taken into account. In particular, 
the functioning of the multidisciplinary team, i.e. col- 
laboration and coordination, is only briefly mentioned 
in the PACIC. Additional team-functioning items would 
increase the content validity because the CCM assumes 
that interventions on the practice level aim to improve 
the functioning of the pro-active multidisciplinary team 
and thereby the quality of chronic care management. 

Third, as far as we know, it has not yet been deter- 
mined if and how the PACIC could be used to com- 
pare the quality of chronic care between GP practices. 



Dutch GP practices, including GPs and GP practice 
nurses working at the same address, provide diabetes 
care collaboratively. Patients' experiences of chronic 
care management within a GP practice could be useful 
as a marketing tool [5]; however, it is unknown whether 
the PACIC identifies differences in patients' experi- 
ences between GP practices. As patients' perspective 
ratings on the quality of chronic care could only be reli- 
ably interpreted by case-mix adjustment, insight into 
the influence at the GP practice level and individual 
characteristics is needed [14, 15]. 

Although the PACIC is considered to be the most appro- 
priate instrument to measure patients' chronic care 
experience [9], several questions need to be answered 
to improve its usefulness. The objectives of this study 
are the following: 1) to assess the psychometric qual- 
ity of the PACIC using the appropriate psychometric 
tests for ordinal data; 2) to assess the psychometric 
quality of the PACIC+, that is, the PACIC including six 
additional multidisciplinary team functioning variables; 
and 3) to test the ability of the PACIC and PACIC+ to 
discriminate between GP practices. 

Theory and methods 
Study population 

Data were obtained from an observational study evalu- 
ating the effects of a bundled payment system for dia- 
betes care in the Netherlands. Details about this study 
are reported elsewhere [16]. For the observational 
study, ten different care groups were selected based on 
size, catchment area, geographical location and com- 
position (e.g. rural vs. urban), and their organisational 
structure. Care groups are legal entities — formed by 
multiple care providers often exclusively GPs — which 
operate as contracting entities to cover a full range of 
diabetes care services for a fixed period. Care groups 
can decide to either deliver the various diabetes care 
components themselves or subcontract other care 
providers [17]. The characteristics of the included care 
groups are outlined in Appendix 1 . 

As part of the observational study, a questionnaire was 
sent to a random sample of 4377 diabetes patients 
clustered within a random sample of 78 GP practices 
representing eight care groups. The people that receive 
their diabetes care by the GP practices are predom- 
inantly people with diabetes type 2. The goal of the 
questionnaire was to assess the patients' experiences 
with chronic care, and it incorporated questions about 
demographic and clinical patient characteristics, the 
PACIC+, and patient outcomes. The first three authors 
sent the questionnaires to the patients. After three 
weeks, reminders were sent to non-respondents. 
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Measures 

The PACIC was used to identify the extent to which 
the chronic care was congruent with Wagner's CCM in 
the past 12 months. The PACIC consists of 20 ques- 
tions with response categories ranging from 1 'almost 
never' to 5 'almost always', with higher scores indi- 
cating a higher extent to which patients received inte- 
grated care following the CCM elements [8]. We used 
the Dutch PACIC translated by Vrijhoef etal. [9]. Glas- 
gow et al. identified five subscales of the PACIC: 1) 
Patient activation (3 items), 2) Delivery system design/ 
Decision support (3 items), 3) Goal setting (5 items), 
4) Problem solving/Contextual counselling (4 items), 
and 5) Follow-up/Coordination (5 items) [8]. The sub- 
scores for each scale were computed by averaging 
across items completed within that scale, and the 
overall PACIC was scored by averaging scores across 
all subscales. 

Furthermore, the PACIC was upgraded by including 
six additional questions regarding multidisciplinary 
team functioning, i.e. collaboration and coordination, 
which are used in the Dutch consumer quality index 
(CQ-index) instrument [18, 19] and the Dutch panel 
of chronic illnesses [20]. The CQ-index instruments 
were developed in the Netherlands to assess the qual- 
ity of care based on the American Consumer Assess- 
ment of Health care Providers and Systems (CAHPS) 
and the Dutch Quality of Care Through the Patient's 
Eye (QUOTE) [14, 18]. The scores on these six addi- 
tional items have identical response categories and 
scores as the PACIC. The 20 items of the PACIC and 
the 6 additional items of the PACIC+ are outlined in 
Appendix 2. 

The demographic and clinical patient characteristics 
included in the study were age, sex, ethnicity, educa- 
tional level, type of diabetes, duration of diabetes, and 
co-morbidity. Ethnicity was defined as Western (North- 
America, EU (except Turkey), Japan, Indonesia) and 
non-Western, and education was defined as low (lower 
vocational or less education), middle (secondary edu- 
cation), and high (higher education). 

Analysis 

Descriptive analyses were applied to describe the 
baseline characteristics of our study population. The 
psychometric quality of the PACIC and PACIC+ was 
measured by reliability and factor analysis. The reli- 
ability was tested by assessing the internal consisten- 
cies with the Cronbach's alpha. A Cronbach's alpha 
of 0.80 or higher was accepted as a good score [21]. 
The factor analysis included an explorative factor 
analysis (EFA) and confirmatory factor analysis (CFA) 
using the split-half method. After splitting the data-file 



randomly, we performed EFA with the first half. Three 
types of EFA were applied to identify the factor struc- 
ture. Principal axis factoring (PAF) with oblimin rota- 
tion was first performed to explore the factor structure 
of the data. Subsequently, parallel analysis (PA) was 
performed to identify the number of factors following 
the O'Connor's SPSS syntax, which is suitable for 
ordinal data. Lastly, Velicer's MAP test was applied 
with O'Connor's syntax to perform a factor analysis 
based on the polychoric correlation matrix obtained 
in R [22]. Subsequently, CFA was performed in R with 
the second half of the data to confirm the hypotheses 
concerning the underlying structure generated by the 
EFA as well as the factor structure determined by 
Glasgow. CFA tests the correlation structure of the 
data against the hypothesised structure and rates 
the 'goodness of fit' [21, 23]. A value >0.10 of the 
root-mean-square-error-of-approximation (RMSEA) 
fit index indicates an unacceptable fit of the model, 
which implies that the correlation between the items 
within the tested factor structure might be a coinci- 
dence [21]. 

We tested the ability of the questionnaire to dis- 
criminate between GP practices by performing mul- 
tilevel analysis in SPSS [24] because differences in 
PACIC scores were expected based on the fact that 
differences in chronic care management are preva- 
lent [25]. A GP practice was defined as one or more 
GPs working at the same location. The Intra-Class 
Correlation (ICC) was computed to identify which 
part of the total variance in the outcomes could be 
attributed to the difference between GP practices. 
We examined the influence of GP practice level 
before (null model) and after case-mix adjustment 
(full model). 

Missing value imputation 

Complete case analysis was applied because the 
PACIC is most frequently analysed using complete 
case analysis [9, 11, 26-28]. However, because mul- 
tiple imputation offers advantages over complete case 
analyses [29, 30], we also performed the analysis 
with multiple imputation (Ml) datasets. Missing val- 
ues were imputed using the mice (multiple imputation 
by chained equation) procedure in R [31]. The mice 
procedure assumes that the distribution of each vari- 
able given the others can be modelled with a logistic 
regression if the variable is dichotomous, polytomous 
logistic regression if it is categorical with three or 
more categories, or linear regression if it is continuous 
with predictive mean matching [32]. The Gibbs sam- 
pler was used to determine the necessary number of 
iterations to compute 20 imputations for each missing 
value [31]. 
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Results 

Patient characteristics 

A total of 1941 patients returned the questionnaire 
(46% response rate) (Figure 1). Of them, 1547 (80%) 
completed all PACIC items, while at least one of the 
items was not completed by the remaining number of 
patients (20%). In total 967 patients completed the six 
additional items of the PACIC+. 

Respondents that fully completed the PACIC differed 
from those who had one or more missing response 
(Appendix 2). Hence, the missing values were multiply 
imputed using 100 iterations. The main characteristics 
of the study population are outlined in Table 1. Almost 
all patients were Western (95%) and had type 2 diabe- 
tes (93%). 

Psychometric quality of the PACIC and 
the PACIC + 

Reliability 

The internal consistency of the items within the PACIC 
was good. Cronbach's alpha was 0.916 with complete 
case analysis and 0.919-0.920 for Ml datasets. The 



internal consistency of the PACIC+ was comparable 
to the PACIC's internal consistency. Cronbach's alpha 
was 0.907 for complete cases and 0.909-0.913 for Ml 
datasets. 

Explorative factor analysis (EFA) 

The EFA results were diffuse for the PACIC as well as 
for the PACIC+. The three EFA methods— PAF, PA, and 
Velicer's MAP — identified varying number of factors for 
the PACIC and the PACIC+. The PAF, PA, and Velicer's 
MAP test with the complete cases identified 4, 8, and 2 
factors, respectively, for the PACIC. The PACIC+ had 
5, 6, and 3 factors, respectively. EFA on the Ml data- 
sets showed comparable inconclusive results. These 
results imply that no specific subscales of the PACIC 
and the PACIC+ can be inferred with confidence. 

Confirmatory factor analysis (CFA) 

Because different factor structures were suggested by 
the different EFA methods, we tested all factor struc- 
tures proposed by PAF-analysis if at least 2 items were 
included per factor. In addition, we tested the factor 
structure proposed by Glasgow et al. [8]. All models 
gave a p-value below 0.001 , which implies that the null 
hypothesis of perfect fit is rejected. We assessed the fit 
using RMSEA, but none of the models had an accept- 
able fit (all models had a RMSEA over 0.10) (Table 2). 



Questionnaires sent: n=4377 

Died: 9 
Moved: 1 1 

n= 112 Returned sender: 13 

Wrong address: 17 
No diabetes: 5 
No diabetes treatment: 57 



Net questionnaires sent: n=4265 





n=2324 




No response: 2276 
Empty questionnaires: 35 
Responded too late: 13 







Questionnaires returned: n=1941 





n=394 




1 missing response: 85 

2 missing responses: 33 

3 or more missing responses: 276 













PACIC completed: n=1547 



n=580 



1 missing response: 45 

2 missing responses: 22 

3 or more missing responses: 513 



PACIC+completed*: n=967 



Figure 1. Questionnaire response. 

•Respondents that completed the PACIC and the 6 additional multidisciplinary team functioning questions of the returned questionnaire. 
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Table 1. Patient characteristics 





Questionnaire 




DAPIP4. 




returned 3 


completed b 


completed 0 




n=1941 


n=1547 


n=967 


GP practices characteristics 








Number of GP practices 


78 


69 


67 


Mean number of work-experience: mean (SD) 


19.0 (7.1) 


18.9 (6.9) 


19.0 (6.8) 


Percentage female GPs: mean (SD) 


44.3 (28) 


45.3 (27) 


45.5 (27) 


Size: mean (SD) 


3.7 (1.8) 


3.8 (1.8) 


3.9 (1.7) 


Patient characteristics 








Age in years: mean (SD) 


67.0 (11.4) 


65.7(11.4)* 


64.5(11.5)* 


Male: % 


50.1 


53.6* 


56.4* 


Education level: % 








Low 


44.3 


40.2 


37.9 


Middle 


40.1 


42.1 


43.8 


High 


15.6 


17.7 


18.3 


Western: % 


95.7 


95.4 


95.3 


Duration of diabetes in years: mean (SD) 


9.4 (8.8) 


9.5 (8.7) 


9.8 (8.8) 


Diabetes type 2: % 


93.4 


93.0 


92.7 


Co-morbidity: % 


59.8 


60.2 


62.2* 



a) Patients who returned the questionnaire, not all patients completed the questionnaire fully; b) patients who completed the PACIC without missing responses; 
c) patients who completed the PACIC+ without missing responses; 'Significant difference compared to patients that returned the questionnaire. 



Discriminant capacity between GP 
practices 

The results of the multilevel model analysis that con- 
sidered patients at first level and GP practices at sec- 
ond level are outlined in Table 3 for the Ml data. The 
null model of the multilevel analysis with PACIC and 
PACIC+ scores showed an ICC of 0.052 and 0.051, 
respectively (Table 3). These results imply that 5.2% 
and 5.1% of the total variance of the PACIC and 
PACIC+ scores can be attributed to variation on GP 
practice level. Regarding the patient characteristics, 
higher age and higher education levels were asso- 
ciated with lower PACIC and PACIC+ scores. The 
models with case mix adjustment showed an ICC of 
0.057 and 0.054, respectively. Complete case analysis 
showed no substantial differences (Appendix 3). 



Table 2. Results of CFA with factor structure proposed by PAF 
analysis 





PACIC 




PACIC+ 




Number of 


CCA a 


Ml datasets b 


CCA a 


Ml datasets" 


factors 


RMSEA 


RMSEA 


RMSEA 


RMSEA 


One 


0.132 


0.125-0.128 


0.129 


0.124-0.130 


Two 


0.120 


0.116-0.126 


0.109 


0.106-0.112 


Three 


0.112 


0.109-0.117 


0.107 


0.103-0.109 


Four 


0.117 


0.112-0.123 


0.105 


0.103-0.109 


Five 


0.111 


0.110-0.142 


0.103 


0.099-0. 103 c 


Six 


NA d 


NA d 


NA d 


NA d 



a) CCA, complete case analysis; b) Ml datasets, analysis of the first 5 mul- 
tiple imputated datasets; c) RMSEA<0.10 in only one Ml dataset; d) NA, not 
assessed since the number of variables in one factor was <2. 



Discussion 

This study examined the PACIC to improve and vali- 
date its potential to measure the experience of diabe- 
tes patients in daily chronic care practice. The results 
revealed that the reliability of the PACIC is good to mea- 
sure chronic care experience of patients with diabetes. 
Moreover, our results revealed that the reliability of the 
extended PACIC (PACIC+), which also included team 
functioning, was also good. No specific subscales of the 
PACIC and the PACIC+ can be inferred with confidence. 
Furthermore, the PACIC and PACIC+ identified varia- 
tion in patients' experience between GP practices. 

This was the first study that assessed the psychometric 
quality of the PACIC by applying the appropriate statis- 
tics for the original scale structure proposed by Glas- 
gow et al. [8]. Our results regarding the psychometric 
quality (reliability and no inferred subscales) are in line 
with the results of Gugiu which performed a valida- 
tion study with another unsuccessful PACIC response 
category [12]. Furthermore, we could not infer a spe- 
cific factor structure with confidence, which implies 
that previous identified subscales should not be used 
for quality improvement efforts. Patients chronic care 
management experiences scores of the PACIC should 
only be used as overall score or be related to one of 
the 26 specific items to prioritize quality improvement 
goals. The five factors suggested by Glasgow were 
only identified when we applied the inappropriate Pear- 
son correlations that were used in earlier validation 
studies (data not shown). Thus, differences between 
our results and previous validation studies that ignored 
the ordinal data structure could be explained by the 
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Table 3. Multilevel analysis results with multiple imputed data 





PACIC 




PACIC+ 






Null model 


Full model 


Null model 


Full model 




P (p-value) 


p (p-value) 


fy /an*. Hill m\ 

[5 (p-value) 


p (p-value) 


Intercept 


2.786 (0.000) 


3.339 (0.000) 


3.058 (0.000) 


3.494 (0.000) 


Age 




-0.008 (0.000) 




-0.006 (0.002) 


Male 




0.025 (0.593) 




0.005 (0.896) 


Educational level 3 










Low 




0.098 (0.063) 




0.060 (0.184) 


High 




-0.147 (0.028) 




-0.129 (0.025) 


Non-Western 




-0.049 (-0.397) 




-0.103 (0.337) 


Duration of diabetes 




-0.003 (0.287) 




-0.003 (0.176) 


Type 1 diabetes b 




-0.110 (0.276) 




-0.123 (0.153) 


Comorbidity 




-0.002 (0.966) 




0.032 (0.444) 


Variance patients 


0.675 (0.000) 


0.647 (0.000) 


0.487 (0.000) 


0.471 (0.000) 


Variance GP practices 


0.037 (0.001) 


0.039 (0.003) 


0.026 (0.003) 


0.027 (0.005) 


ICC 


0.052 


0.057 


0.051 


0.054 



a) Reference is middle education level; b) reference is type 2 diabetes, ICC, intra-class correlation. 



difference in methods. Hence, the previously incor- 
rectly identified five-factor structure should be used 
with caution because of methodological limitations. 

Furthermore, in addition to the study by Gugiu, we 
extended the PACIC by including multidisciplinary team 
functioning items, which are essential for the qual- 
ity of chronic care based on the chronic care model 
[33]. This study showed good reliability scores of the 
PACIC+. This result implies that the PACIC+ can be of 
additional value without a loss of psychometric quality 
compared with the PACIC. 

In addition to previous validation studies, this study 
evaluated the extent to which the variation of patients' 
chronic care experience was related to the GP practices 
from which they receive their chronic diabetes care. 
The influence of this so-called 'second level' has only 
been studied once, by Glasgow and colleagues, and 
they identified the physician influence [10]. In line with 
other patient experience questionnaires, our results 
revealed that both the PACIC and the PACIC+ could be 
used to compare GP practices. Although the discrimi- 
nator capacity of the GP practices can be expected to 
differ between settings [34], its magnitude was equiva- 
lent to or even slightly higher than previous patients 
experiences studies revealed [14, 34, 35]. In addition, 
the potential discriminator capacity is expected to be 
higher because we have selected a relatively homoge- 
nous group; all included GP practices already cooperate 
within one of the recently introduced cooperatives (i.e. 
care groups). In other words, the included GP practices 
were frontrunners regarding chronic care management 
at the start of our study. Future research should study 
the influence of this recently introduced third level, the 
care group, as these groups aim to improve the qual- 
ity of care of their GP practices. Because the PACIC 
and PACIC+ were applicable as benchmark tools, our 



results also revealed that case-mix adjustment was 
needed for both instruments. 

The PACIC and PACIC+ were validated with a sub- 
stantial number of respondents to gain insight about 
a useful tool for professionals, policy-makers and 
researchers to assess and improve the quality of 
diabetes care. Nevertheless, this study had several 
limitations. First, we validated a disease-specific 
instrument to assess patients' chronic care experi- 
ence. However, it can be questioned whether this 
instrument should be used every year to identify the 
quality of chronic care for specific diseases (in this 
case diabetes) or if it should be transformed into a 
generic chronic care instrument. Second, we could 
not preclude social desirable answers. Though, 
social desirability is likely to be restricted as the first 
three authors sent the questionnaires directly to the 
patients without interferences of the GPs. The poten- 
tial influence of GPs attitude or willingness to attain 
high patient experience scores will also be limited 
as results were not reported to the GP practices at 
GP practice level. Third, there was a relatively low 
response rate (46%). Analysis of non-responders 
was not possible, but the patient characteristics of the 
study population were not substantially different from 
those of the patients with diabetes included in the 
diabetes care program within the eight care groups 
(data not shown). In addition, differences persisted 
between the patients that fully completed the ques- 
tionnaire and patients who had one or more missing 
response. Although we applied multiple imputation 
to address the missing data [29, 30], improvement 
of the questionnaire for future research would better 
reduce the differences in response rates regarding 
these patient characteristics: age, educational level, 
sex and co-morbidity. 
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Although the PACIC is already widely used, further 
steps to evaluate chronic care management experi- 
ences of patients are needed. First, additional qualita- 
tive research is needed to improve the response rate. 
This will require an understanding of why respon- 
dents did not answer certain questions in the PACIC. 
Second, the Dutch PACIC, and possibly also the 
other versions [8, 36-38], may need some improve- 
ment regarding several items that are not applicable 
for all patients and/or could be achieved in another 
way. For instance, 'referring patients to a dietician or 
social worker every year' is not necessary for all dia- 
betic patients according to the guidelines, and 'get- 
ting a copy of the treatment plan' could be achieved 
in another way because web-based IT systems are 
being implemented. Because some of these changes 
will not be country specific [37], consensus should 
be achieved internationally to ensure uniformity for 
future comparisons between countries. Third, we 
extended the PACIC with six questions to incorporate 
CCMs team functioning component. Although these 
six questions are frequently applied in Dutch patient 
questionnaires and translated into English via the for- 
ward/backward method, validation of these translated 
questions could be of additional value. Fourth, the 
association between PACIC scores and patient out- 
comes needs to be studied. Because many countries 
shape their health system in response to chronic care 
model assumptions — such as the Accountable Care 
Organisations (ACOs) in the US, the Clinical Commis- 
sioning Groups (CCGs) in England, and Care Groups 
(CGs) in the Netherlands — insight into its effective- 
ness and patient-centred effect measures are greatly 
needed. 

Conclusions 

The PACIC and the PACIC+ are both reliable instru- 
ments to measure the chronic care management expe- 
riences of patients with diabetes. Previously identified 
subscales should be used with caution. The PACIC+ 



should be preferentially used because it also includes 
multidisciplinary team functioning, which is one of 
the central pillars of chronic care management. Both 
PACIC instruments can be used to identify variation in 
chronic care management between GP practices. Nev- 
ertheless, it is advisable to further improve the PACIC+ 
to increase the content validity and response rates. 
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Appendix 1 



Legal form Care groups 





1 


2 


3 


4 


5 


6 


7 


8 




Cooperative 


Foundation 


LLC 


LLC 


Limited 


Foundation 


LLC 


LLC 












partnership 








Start BP contract 


2007 


2007 


2007 


2007 


2009 


2007 


2007 


2010 


Number of patients in DM-care 
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11,000 


3400 


13,500 


6500 


6800 


4600 


3600 


program 


















Number of participating GPs in 


6 
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130 
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DM-care program 


















Number of participating GP practice 
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18 
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35 


30 


35 
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nurses in DM-care program 


















Interagency patient files 
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No 
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Yes 


Multidisciplinary protocol 
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Yes 
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Yes 


Reflective information 


Yes 


Yes 
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Yes 


Patient recall system for no-shows 


Yes 


Yes 


Yes 


Yes 


No 


Yes 


Yes 


Yes 


Patient portal 


No 


Yes 


No 


No 


Yes 


No 


No 


No 
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Appendix 3 





PACIC (n=1547) 




PACIC+(n=967) 




Null model p 
(p-value) 


Full model (3 
(p-value) 


Null model (3 
(p-value) 


Full model (3 
(p-value) 


Intercept 


2.769 (0.000) 


3.137 (0.000) 


3.090 (0.000) 


3.400 (0.000) 


Age 




-0.006 (0.015) 




-0.003 (0.202) 


Male 




0.058 (0.254) 




0.001 (0.993) 


Educational level 8 










Low 




0.103 (0.069) 




0.072 (0.251) 


High 




-0.160 (0.018) 




-0.113 (0.133) 


Non-western 




-0.085 (0.512) 




-0.103 (0.436) 


Duration of diabetes 




-0.005 (0.015) 




-0.006 (0.080) 


Diabetes type l b 




-0.028 (0.785) 




0.002 (0.986) 


Comorbidity 




0.024 (0.646) 




-0.082 (0.152) 


Variance patients 


0.613(0.000) 


0.581 (0.000) 


0.454 (0.000) 


0.442 (0.000) 


Variance GP practices 


0.051 (0.001) 


0.051 (0.001) 


0.028 (0.013) 


0.0342 (0.016) 


ICC 


0.077 


0.081 


0.058 


0.072 
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